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The National Academy of Sciences is a private, nonprofit, self-perpetuating society of 
distinguished scholars engaged in scientific and engineering research, dedicated to the 
furtherance of science and technology and to their use for the general welfare. Upon the 
authority of the charter granted to it by the Congress in 1863, the Academy has a mandate 
that requires it to advise the federal government on scientific and technical matters. Dr. 
Bruce Alberts is president of the National Academy of Sciences. 

The National Academy of Engineering was established in 1964, under the charter of the 
National Academy of Sciences, as a parallel organization of outstanding engineers. It is 
autonomous in its administration and in the selection of its members, sharing with the 
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national needs, encourages education and research, and recognizes the superior 
achievements of engineers. Dr. William A. Wulf is interim president of the National 
Academy of Engineering. 

The Institute of Medicine was established in 1970 by the National Academy of Sciences to 
secure the services of eminent members of appropriate professions in the examination of 
policy matters pertaining to the health of the public. The Institute acts under the 
responsibility given to the National Academy of Sciences by its congressional charter to be 
an adviser to the federal government and, upon its own initiative, to identify issues of 
medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of 
Medicine. 

The National Research Council was organized by the National Academy of Sciences in 
1916 to associate the broad community of science and technology with the Academy's 
purposes of furthering knowledge and advising the federal government. Functioning in 
accordance with general policies determined by the Academy, the Council has become the 
principal operating agency of both the National Academy of Sciences and the National 
Academy of Engineering in providing services to the government, the public, and the 
scientific and engineering communities. The Council is administered jointly by both 
Academies and the Institute of Medicine. Dr. Bruce Alberts and Dr. William A. Wulf are 
chairman and interim vice chairman, respectively, of the National Research Council. 
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Preface 



The Mathematical Sciences Education Board (MSEB) was established in 1985 by the National 
Research Council, to "maintain a national capability for assessing the status and quality of 
mathematics education." Since its inception, the MSEB has been deeply interested in international 
comparative studies which can illuminate public understanding of the U.S. mathematics education 
system. The MSEB sponsored a convocation on "International Comparisons of Mathematics 
Education: Policy Implications for the United States," in conjunction with the Second International 
Mathematics Study, in 1987. It has hosted several presentations on the recently completed Third 
International Mathematics and Science Study (TIMSS) at its Board meetings in the past few years. 

In 1995, when the NRC established the Center for Science, Mathematics, and Engineering 
Education, the MSEB was joined by the Committee on Science Education (COSE K-12). In 
response to a request from the National Science Foundation, and with its support, MSEB and 
COSE K-12 have joined to produce this short report, in preparation for the impending release of 
the TIMSS data. It is further accompanied by a brochure, intended for broad public dissemination. 

This report is designed to help frame discussion and analysis of data that will emerge from TIMSS, 
a landmark study of mathematics and science education that investigates the mathematics and 
science curricula at three academic levels (age 9, age 13, and the last year of high school), in more 
than forty countries. Compared to its predecessors, and to other international comparative studies, 
TIMSS is noteworthy both in the number of variables documented and in the sophistication of the 
instruments used. 

The primary source for this report is the first released component of TIMSS, the Survey of 
Mathematics and Science Opportunities (SMSO); it draws also on related published documents, 
and on informal consultation with TIMSS staff. Direct access to the TIMSS data was restricted. 
Accordingly, this report highlights directions for further inquiry more than findings. It should not 
be construed as a review of the SMSO design and methodology, although we expect that such 
review and critique will be undertaken by the field. 

SMSO, a study of six of the TIMSS countries, was intended to develop and pilot the innovative 
survey instruments and international comparison frameworks employed by TIMSS. These 
frameworks distinguish three expressions of curriculum: the intended curriculum is examined 
through textbooks and lists of goals and objectives formulated at national, state, or local levels; the 
implemented curriculum or the patterns of instructional practice, is studied using questionnaires 
and, on a smaller scale, classroom observation; the attained curriculum is measured through 
student achievement data. The development and piloting process undertaken by SMSO also 
revealed some fundamental, and sometimes surprising, subtleties that emerge in making 
international comparisons. 

The value to an individual country of international comparative studies is not so much to furnish a 
ranking, but rather to provide a broad knowledge base with which to understand and improve its 
own condition. To serve this end, international educational studies must provide, in addition to 
rankings of achievement, contextual data that permit analysis and interpretation of the observed 
outcomes. Imagine for example, international statistics on mortality rates, or the epidemiological 
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patterns of certain diseases. Such data would be of limited use if not accompanied by country- 
specific information about diet, lifestyles, physical environment, social institutions, and so on. 

What is most significant about TIMSS is the degree to which it will furnish profiles of curricular 
and instructional practices, thereby providing the context for interpreting its achievement data. In 
place of the premature and inappropriate conclusions sometimes drawn from past achievement 
data, TIMSS presents an unprecedented opportunity to initiate informed analysis and interpretation 
of the culturally diverse conditions and practices that have produced its achievement results. A 
primary purpose of this report is to help promote this stance for the reception of the TIMSS data 
by the various communities concerned with educational improvement in the U.S. TIMSS and 
SMSO do not furnish conclusions ; rather they invite and enable fruitful inquiry. This report 
indicates some such directions of inquiry that emerge already from SMSO. 

There are moments when the conditions that favor responsible scholarship - thorough and unrushed 
analysis and synthesis of existing data and research - brush against the expediencies of engagement 
with real events in real time. The production of this report represents such an occasion. The 
precipitating event is the impending release of the TIMSS data, and the attendant potential for 
widespread misinterpretation. Enthusiasm for international comparisons creates a pressure for fast 
reporting of summary data, without crucial analyses. Efforts to improve mathematics and science 
education in the U.S. are now intense, and their aims and effects are widely debated. The TIMSS 
data have the potential to inform these debates and to contribute to these efforts at improvement. 

By the same token, inappropriate uses of the data have the potential to impede the progress of 
educational improvement. The public, eager to know how the United States compares with other 
countries, is prone to draw quick conclusions without adequate analysis or discussion. This danger 
is what gives urgency to the central message of this letter report. If TIMSS is to provide the kinds 
of resources for educational improvement of which it is capable, careful analyses of the relations 
among critical variables will be crucial. We seek in this document to orient others to this important 
study, and to begin the process of questioning and analysis that can most effectively contribute to 
educational improvement. 

In order for this report to most effectively serve its public information purpose, it had to be 
available in close proximity to the release of the TIMSS data. This required that it be executed 
with extraordinary dispatch. It is a tribute to the writing group assembled that this was 
accomplished. Joan Ferrini-Mundy deftly organized and conceptualized this effort; she further 
assured, under conditions of limited access to source materials, the scholarly rigor of the report. 

She was greatly aided by generous and dedicated staff support from Ramona Irvin. It is unlikely 
that this would have reached successful conclusion in timely fashion without the extraordinary 
intellectual leadership and writing skills of Deborah Ball, to whom we all are deeply indebted. 

Hyman Bass, Chair Jane Butler Kahle, Chair 

Mathematical Sciences Committee on Science 

Education Board Education K-12 
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Mathematics and Science Education Around the World: 

What Can We Learn from the Survey of Mathematics and Science Opportunities (SMSO) and 
the Third International Mathematics and Science Study (TIMSS)? 

Overview 

Amid current efforts to improve mathematics and science education in the United States, people often 
ask how these subjects are organized and taught in other countries. They hear repeatedly that other 
countries produce higher student achievement. Teachers and parents wonder about the answers to 
questions like these: Why do the children in Asian cultures seem to be so good at science and 
mathematics? How are biology and physics taught in the French curriculum? What are textbooks 
like elsewhere, and how much latitude do teachers have in the way they follow the texts? Do all 
students receive the same education, or are they grouped by ability or perceived educational promise? 
If students are grouped, how early is this done? What are tests like, and what are the consequences 
for students? Are other countries engaged in Standards-like reforms? Does anything like "standards" 
play a role in other countries? 

At times, international comparisons are framed not as questions but as assertions, such as "The Asians 
cover many more topics"; "The Japanese are in school longer"; "The French require more 
homework"; "High levels of instant recall and speed in calculations are required of Chinese students." 
These assertions reflect widely shared beliefs about education in other countries. Although these 
claims are quite common, their factual bases vary. They indicate the high level of attention to 
mathematics and science education around the world. 

Naturally, interest is greatest about countries whose students seemingly outperform U.S. students. 
Sometimes this intense interest stems from a desire to identify those practices that are associated with 
student achievement, characterized by the question, "What are other countries doing that works?" 
Sometimes the interest is rooted in a defensiveness, expressed as "They only score higher because 
they exclude so many of their students by high school, while we educate all our students" or as "The 
pressure on their students is excruciating." 

Questions such as these reflect more than a casual interest in other countries' educational practices. 
They grow out of an interest in identifying ways to improve mathematics and science education in the 
United States. 

In this report, we introduce the Third International Mathematics and Science Study (TEMSS), 1 a 
major international investigation of curriculum, instruction, and learning in mathematics and science, 
which will provide resources for addressing these and other important questions. We draw on a pilot 



’TIMSS is a collaborative research project sponsored by the International Association for the Evaluation of 
Educational Achievement (IEA). The IEA will release a series of international TIMSS reports beginning in 
the fall of 1996. Several U.S. TIMSS reports, including detailed reports on findings and methodology, also 
will be released beginning at this time. Dr. Albert Beaton, Boston College, is the TIMSS International Study 
Director. Dr. William Schmidt, Michigan State University, is the U.S. National Research Coordinator. 
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project related to this study, the Survey of Mathematics and Science Opportunities (SMSO) 2 , 3 . 

SMSO describes the development and piloting of the TIMSS instruments and methodology and 
reports preliminary findings. 

The focus of this short report is on what TIMSS will be able to contribute to understandings of 
mathematics and science education around the world as well as to current efforts to improve student 
learning, particularly in the United States. The report centers on three sets of questions: 

• What is the Third International Mathematics and Science Study (TIMSS) ? How is opportunity to 

learn viewed in TIMSS? What kinds of information have the researchers collected? What are 
some of the challenges and opportunities of cross-national work? 

• What is the Survey of Mathematics and Science Opportunities (SMSO) ? What can be learned from 

it? What does SMSO say about intended curriculum? What does SMSO say about the 
implemented curriculum and instructional practices? 

• What questions might be explored with TIMSS? What do the preliminary issues in SMSO suggest 

about questions that should be pursued in the TIMSS data? What issues raised by this first study 
are important to bear in mind in interpreting TIMSS findings? What are the implications for 
secondary analysis? 

The release of results from TIMSS, beginning in 1996-1997, will attract significant interest and 
attention. A study such as this is a complicated undertaking. Factors that may vary dramatically 
across countries include educational objectives and examinations. There are important differences in 
how educational objectives and examinations are determined, how they influence teaching and 
learning, and how they are viewed. The TIMSS data, which focus not only on student achievement 
but also on educational systems and practices, will make it possible to ask questions about and 
analyze factors that combine to influence educational outcomes. 

Due to substantial developments in methods of international comparison as well as the ambitious 
scope of the TIMSS study, the analyses possible with the TIMSS data set will be unprecedented. The 
study is also an object of concern for some observers. Issues of quality control in sampling, 
translation of instruments, and collection and management of data are sometimes problematic in 
international studies. 4 The task of this report is not to assess TIMSS or SMSO. The report neither 



2 The SMSO project began in 1991 as a collaborative effort to understand the key elements of teaching and 
learning in mathematics and science classrooms in six countries. SMSO was intended to develop theoretical 
models and methodological tools to inform the design of TIMSS. SMSO is described in Schmidt, et al. 
(19%). Characterizing pedagogical flow: An investigation of mathematics and science teaching in six countries. 
Dordrecht, The Netherlands: Kluwer Academic Publishers. 

3 TIMSS and SMSO are funded by the National Science Foundation, the National Center for Education 
Statistics, the Canadian Ministry of Human Resources Development, and participating countries. 

4 Board on International Comparative Studies in Education. (1993). A collaborative agenda for improving 
international comparative studies in education. Washington, DC: National Research Council. 
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endorses nor critiques this research. As these data about curriculum, teaching, and student 
achievement emerge, the methods of the work will be criticized, questioned, and lauded. Written 
before the release of the TIMSS data and hence too early for a careful review of the study or its 
products, this report aims instead to inform the reader about TIMSS and to foreshadow the kinds of 
questions, information, and interpretations that may follow. The scope of the study is large; the 
challenge to make sense of the information great. The utility, reliability, and validity of the 
information and interpretations that TIMSS will produce remains to be seen. 

This report is intended to set the stage for discussions and constructive utilization of an undeniably 
extensive data set about mathematics and science education. We do this in two ways. First, we 
present a summary of the TIMSS research framework and methods, then we highlight preliminary 
findings from SMSO about curriculum and teaching and point to questions raised by these findings. 



What Is the Third International Mathematics and Science Study (TIMSS)? 

Beginning in 1996-97, data from the Third International Mathematics and Science Study (TIMSS), 
will be released. This ambitious cross-national study measures student achievement in 44 countries 
(see Appendix 1). In addition, extensive data have been gathered about conditions of schooling, 
curriculum, and instructional practice. The ultimate potential of TIMSS is to be able to place each 
country's student achievement data within a sufficiently complex portrait of that country's educational 
system to enable identification, examination, and analysis of important relations among factors. Such 
analyses are essential to informed judgments about ways to improve education. 

TIMSS will report on the mathematics and science achievement of students enrolled in the two 
adjacent grades that contain the largest proportion of 9-year-olds, the two adjacent grades that 
contain the largest proportion of 13-year-olds, and students in their final year of secondary education. 
All countries participated in testing 13-year-olds; testing at other grade levels was optional. The 
achievement data were gathered primarily in the 1994-95 academic year. See Appendix 4 for data 
release plans. 

How Is Opportunity to Learn Viewed in TIMSS? 

The TIMSS design reflects the importance of analyzing relationships between opportunities to learn 
and educational outcomes. The three basic elements of TIMSS are the intended curriculum (the 
educational system's aims and goals); the implemented curriculum (the actual strategies, practices, 
and activities found in classrooms); and the attained curriculum (student learning). Consistent with 
this basic structure, the TIMSS perspective on educational experience includes school variables and 
processes and a view of student learning influenced by psychological and sociological perspectives. 
The TIMSS conceptual model (Appendix 2) provided a theoretical foundation for collecting data and 
an initial approach to examining relationships among findings. Four questions are at the heart of the 
design. One central question, "What are students expected to learn?", corresponds to the intended 
curriculum. A second question adds a focus on teachers — "Who delivers the instruction?" The third 
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question represents the implemented curriculum, "How is instruction organized?", while the fourth, 
"What have students learned?", corresponds to the attained curriculum. 

What Kinds of Information Have the TIMSS Researchers Collected? 

In order to make sense of the TIMSS data and analyses, it is important to know how the researchers 
conceptualized — and then investigated — several elements of educational systems. Analyses that 
explore connections among these elements are likely to be most productive in enabling valid 
interpretations and conclusions. For instance, information gathered on the content and use of 
textbooks provides a view of the curriculum that is complementary to data gathered on curriculum 
frameworks and tests. Knowing what is in textbooks and how widely they are used or whether and 
how they are even required offers insight into the potential influence of curriculum material on what 
students have opportunities to learn. Yet such information does not necessarily provide insight into 
the curriculum that students encounter in the classroom. Teachers’ decisions about the organization 
and presentation of material also shape the curriculum. And knowing what students are taught still 
would not tell us what any particular student experiences, as mediated by his or her own 
interpretations. All these differences in meaning of curriculum highlight the interface between 
intended and implemented curriculum. 

There are many different ways to interpret what is meant by curriculum. No one is somehow "right"; 
instead, the interpretation depends on the questions one is trying to answer about curriculum. In 
examining data, it is important to understand how curriculum is being defined. What did the TIMSS 
mean by, and how did researchers gather data on, documents and texts — the intended curriculum? 
On instructional practices — the implemented curriculum? On student learning — the attained 
curriculum? Understanding their meanings is crucial to interpreting the TIMSS findings. 

The intended curriculum: TIMSS focused on three elements of the intended curriculum. 
First, researchers gathered and examined a variety of documents from each country that 
offered clues to the curriculum, as it was publicly defined. 5 In many countries, the intended 
curriculum is articulated nationally, through framework documents. In some countries, such 
as the United States and Switzerland, the curriculum is for the most part specified at the state, 
district, or canton 6 — not the national — level. In all cases, the researchers read and analyzed 
curriculum guides and lists of objectives. 

Textbooks comprised a second element of the curriculum examined by TIMSS researchers. 
Researchers assembled a collection of mathematics and science textbooks from each country. 
Approximately 1,600 teachers' guides and textbooks were analyzed across participating 
countries. Wherever possible, the materials were selected to represent those used by a 
majority of a given nation's students. 



5 The publication dates of these materials ranged from 1976 to 1994. The set of U.S. materials was dated 
between 1978 and 1994. 

6 A canton in Switzerland corresponds roughly to a state in the U.S. 
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